I 


O 


AD-A270  841 


Final  Report  for  the  Air  Traffic 
Control  (ATC)  Evaluation  of  the 
Prototype  Terminal  Doppler 
Weather  Radar  (TDWR)  System 


Baxter  Stretcher 


September  1993 
DOT/FAA/CT-TN93/20 


□TIC 

i-!..C:CTE  j 

OCT  13 1993 


D 


Document  is  on  file  at  the  Technical  Center  Library, 
Atlantic  City  International  Airport,  NJ  08405 


u  s  Deportment  of  TransportatKyi 

Federal  Aviation  Administration 

Technical  Center 

Atlantic  City  International  Airport.  N.J  08405 


93-24239 


NOTICE 

This  document  is  disseminated  under  the  sponsorship 
of  the  U.S.  Department  of  Transportation  in  the  interest  of 
information  exchange.  The  United  States  Government 
assumes  no  liability  for  the  contents  or  use  thereof. 

The  United  States  Government  does  not  endorse 
products  or  manufacturers.  Trade  or  manufacturers' 
names  appear  herein  solely  because  they  are  considered 
essential  to  the  objective  of  this  report. 


R«P«rf  N«. 


D0T/FAA/CT-TN93/20 


4.  T t  fl«  and  Subti 


i 


«f¥arnmanr  Aecdttton  No. 


FINAL  REPORT  FOR  THE  AIR  TRAFFIC  CONTROL  (ATC) 
EVALUATION  OF  THE  PROTOTYPE  TERMINAL  DOPPLER 
WEATHER  RADAR  (TDWR)  SYSTEM 


7.  Author'*; 


Baxter  Stretcher 


9.  Ptrf^ming  Organiiotion  Norn*  and  Addraii 

U.S.  Department  of  Transportation 
Federal  Aviation  Administration 
Technical  Center 

Atlantic  City  International  Airport,  NJ  08405 


12-  Sporuortng  Agancy  Nama  aryi  Add.aii 

U.S.  Department  or  Transportation 
Federal  Aviation  Administration 
Technical  Center 

Atlantic  City  International  Airport,  NJ  08405 


Technical  Report  Oocumentetien  Pope 


3.  Racip<«nf'«  Cotoiof  No. 


5.  Roport  Ooto 

September  1993 

6.  Por^ormmg  Orgoniio’ion  Co4d 


6.  Porformmg  Orgonixofion  Ropo^f  No 

DOT/FAA/CT - TN9  3/20 


10.  Work  Unit  No  (TRAIS) 


'  I  Conrroet  O'  Gront  No. 


13.  Typo  of  Ropert  and  Ported  Co^orod 


Technical  Note 


14.  Sponsoring  Agoncy  Codo 


16.  Absfroet 

This  report  details  the  results  of  the  Federal  Aviation  Administration  (FAA) 
operational  evaluation  of  the  Prototype  Terminal  Doppler  Weather  Radar  (TDWR) .  The 
evaluation  was  conducted  by  the  controllers  and  supervisors  at  the  Orlando 
International  Airport  (MCO) ,  Orlando,  Florida.  The  purpose  of  the  test  was  to 
obtain  the  controller's  and  supervisor's  evaluation  of  the  Geographical  Situation 
Display  (GSD)  and  the  Ribbon  Display  Terminal  (RDT)  installed  at  Che  air  traffic 
control  tower  (ATCT) . 

The  responses  from  the  air  traffic  controllers  and  supervisors  led  to  several 
conclusions  concerning  the  prototype  TDWR;  (1)  the  participants  generally  liked  the 
system  and  felt  that  the  prototype  TDWR  was  suitable  and  effective  for  air  traffic 
control  (ATC)  operations,  (2)  the  supervisors,  in  particular,  were  pleased  with  the 
GSD  as  it  helped  them  in  planning  and  making  runway  configuration  decisions  prior 
to  weather  events,  and  (3)  Che  controllers  were  pleased  with  the  products  displayed 
on  the  RDT  and  felt  that  the  RDT  was  effective  and  suitable  for  their  operations. 
There  were,  however,  some  reservations  about  the  size  of  the  RDT  and  the  location 
of  Che  centerfield  wind  on  the  RDT.  The  responses  from  the  controllers  indicated 
that  they  preferred  the  smaller  RDT  (12”  x  11"  x  6")  for  operational  purposes  and 
would  like  to  have  the  centerfield  wind  displayed  in  the  upper  left  corner  of  the 
RDT. 

Based  on  these  conclusions,  two  primary  recommendations  were  made  to  improve  the 
system  prior  to  the  Raytheon  production  TDWR.  The  first  was  that  the  user 
evaluation  of  the  size  of  the  RDT  be  considered  in  the  production  TDWR  and  the 
second  was  the  location  of  the  centerfield  wind  on  the  RDT  be  relocated  to  the  top 
of  the  display. 
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EXECUTIVE  SUMMARY 


The  air  traffic  control  (ATC)  interface  element  of  the  functional  prototype 
Terminal  Doppler  Weather  Radar  (TDWR)  was  evaluated  at  the  Orlando  International 
Airport  (MCO)  during  the  period  May  4  through  July  7,  1992.  The  1992  demonstration 
of  the  prototype  TDWR  was  a  continuation  of  the  operational  demonstration  conducted 
during  the  summer  of  1991.  The  1992  demonstration  evaluated  new  products  and 
algorithms,  in  addition  to  the  existing  products.  Some  of  the  new  products  were 
the  integration  of  the  TDWR  and  the  14  sensor  Low  Level  Wind  Shear  Alert  System 
(LLWAS-3),  storm  tracking,  and  the  effectiveness  of  the  Reflectivity  Attenuation 
Flagging  function. 

The  objective  of  the  evaluation  was  to  obtain  Federal  Aviation  Administration  (FAA) 
air  traffic  controller/supervisor  reaction  to  the  protot3T)e  TDWR  weather  data  and 
display  equipment.  The  displays  consisted  of  four  Ribbon  Display  Terminals  (RDTs) 
and  one  Geographical  Situation  Display  (GSD)  in  the  tower,  and  one  each  RDT  and  GSD 
in  the  Terminal  Radar  Approach  Control  (TRACON) .  Questionnaire  forms  were  used  to 
obtain  responses  from  supervisors  and  controllers  relative  to  operational 
suitability  of  these  displays. 

The  following  are  highlights  of  the  evaluation: 

1.  The  displayed  storm  motion  information  on  the  GSD  was  most  helpful  to  the 
supervisor/controller . 

2.  The  usefulness  of  the  Wind  Shift  Prediction  was  rated  as  good. 

3.  Sixty-four  per  cent  of  the  participants  rated  the  usefulness  of  the  displayed 
microburst  (MB)  information  on  the  RDT  as  good. 

4.  Most  of  the  participants  preferred  the  "small”  RDT  over  the  "large"  RDT  and  the 
centerfield  wind  displayed  in  the  upper  left  corner  of  the  RDT. 

Generally,  the  participants  liked  the  system.  The  supervisors  were  pleased  with 
the  help  they  received  from  the  GSD  in  the  runway  planning  and  configuration 
decisions.  The  controllers  were  pleased  with  the  products  displayed  on  the  RDT  and 
felt  that  the  RDT  was  suitable  and  effective  in  their  operations.  There  were, 
however,  some  reservations  about  the  size  of  the  RDT  and  the  location  of  the 
centerfield  wind  on  the  RDT. 

It  is  recommended  that  if  this  system  is  tested  in  the  future,  the  "small"  RDT 
should  become  the  primary  RDT  for  testing,  and  that  consideration  be  given  to  the 
location  of  the  centerfield  wind  on  the  RDT. 
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INTRODUCTION 


PURPOSE. 

The  purpose  of  this  evaluation  was  to  determine  the  operational  suitability  and 
effectiveness  of  new  products  and  algorithms,  in  addition  to  the  existing  products, 
provided  by  the  functional  prototype  Terminal  Doppler  Weather  Radar  (TDWR) 
hazardous  weather  messages  to  the  controllers  and  supervisors.  Some  of  the  new 
products  were  the  integration  of  the  TDWR  with  the  14  sensor  Low  Level  Wind  Shear 
Alert  System  (LLWAS-3),  storm  motion,  and  the  effectiveness  of  the  Reflectivity 
Attenuation  Flagging  function.  The  evaluation  was  conducted  by  the  Weather  and 
Primary  Radar  Division,  ACW-200D,  of  the  Federal  Aviation  Administration  (FAA) 
Technical  Center.  The  plan  for  this  evaluation  is  detailed  in  the  FAA  Technical 
Center  Plan  for  the  Evaluation  of  the  Prototype  TDWR  System  at  Orlando 
International  Airport  (MCO) ,  Orlando,  Florida,  dated  June  1992. 

BACKGROUND . 

The  TDWR  was  added  to  the  Capital  Investment  Plan  (CIP)  in  1986.  The  project 
consists  of  the  procurement  and  installation  of  47  TDWR  systems  which  will  detect 
microburst  (MB),  gust  fronts  (GF) ,  wind  shifts,  and  precipitation.  The  TDWR  will 
be  used  to  provide  alerts  of  hazardous  weather  conditions  in  the  terminal  area  and 
to  provide  advanced  notice  of  changing  wind  conditions  to  permit  timely  changes  of 
active  runways. 

The  Massachusetts  Institute  of  Technology  and  Lincoln  Laboratory  (MIT/LL)  supports 
the  demonstration  of  the  FAA  prototjrpe  TDWR.  The  HIT/LL  established  a  prototype 
TDWR  test-bed  radar  site  (FL-2)  near  the  Orlando  Airport.  The  site  was  used  to 
develop  operational  algorithms,  collect  experimental  data,  and  develop  user 
friendly  products  for  displays  and  use  by  the  supervisors  and  controllers  in  the 
air  traffic  control  tower  (ATCT)  and  the  Terminal  Radar  Approach  Gontrol  (TRACON) . 
This  demonstration  was  a  continuation  of  the  demonstrations  started  in  the  summer 
of  1990  and  1991  at  Orlando  Airport. 


METHOD 


PARTICIPANTS . 

There  were  50  supervisors/controllers  that  participated  in  this  evaluation 
(39  controllers  and  11  supervisors).  The  Plans  and  Procedure  Specialist  (PPS) 
of  MCO  ATCT  administered  the  questionnaires  to  the  participants. 

TRAINING . 

In  preparation  for  the  demonstration,  MIT/LL  provided  training  to  the  ATCT  and 
TRACON  personnel  in  the  interpretation  and  use  of  the  TDWR  products,  and  in  the  use 
of  the  Geographical  Situation  Displays  (GSDs)  and  the  Ribbon  Display  Terminals 
(RDTs) .  The  PPS  gave  the  supervisors/controllers  a  briefing  prior  to  the 
evaluation  and  the  administration  of  the  questionnaires. 
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The  equipment  that  was  evaluated  consisted  of  four  RDTs  and  one  GSD  in  the  tower, 
and  one  each  RDT  and  GSD  in  the  TRACON. 


RDT. 

The  alphanumeric  products  from  the  TDWR  were  displayed  on  RDTs  located  ac  the 
controller's  position  (circular)  in  the  tower  and  supervisor's  position  in  the 
TRACON.  There  were  two  different  RDTs  sizes:  (1  each)  12"  x  11  1/4"  x  6" 
(small  size)  and  (4  each)  15"  x  15"  x  6"  (large  size) .  The  small  RDT  was 
located  near  the  tower  cab.  During  the  evaluation,  in  lieu  of  the  hazardous 
alert  data  from  the  operational  LLWAS ,  alphanumeric  wind  shear  and  microburst 
alert  messages  from  the  prototjrpe  TDWR  system  were  displayed  on  the  RDTs. 

These  messages  were  displayed  along  with  the  LLWAS  centerfield  and  active 
runway  threshold  winds. 


The  alert  warning  messages  were  displayed  on  the  RDTs  in  the  following  form; 


Runway  ID 

(dir/kts) 

Wind  Shear  Type 

Expected 

loss/gain 

Location 

1st  Encounter 
(kts) 

Threshold 

A  t)rpical 

example  might  be : 

19D 

MBA 

8  OK 

2MD 

320  14 

which  is  read  as;  runway  19  departure,  microburst  alert  (MBA),  expect 
80  knot  loss,  encounter  at  2  miles  on  departure,  runway  threshold  winds 
at  320®  at  14  knots. 


GSD. 

The  tower  and  TRACON  had  a  color  GSD  which  uses  a  Sun  workstation  to  display 
weather  information  to  air  traffic  control  (ATC)  supervisors  and  controllers. 

It  functions  as  a  situation  display  monitor  and  as  an  air  traffic  planning 
tool  for  runway  management.  This  color  workstation  provides  graphical 
representation  of  the  location  and  intensity  of  MBs,  precipitation  cells, 
and  GFs,  as  well  as  estimates  of  the  speed  and  direction  of  motion  for 
precipitation  cells  and  GFs. 

EVALUATION  QUESTIONNAIRE . 

In  order  to  obtain  feedback  from  the  users ,  two  questionnaires  were  developed  by 
the  FAA  Technical  Center.  One  of  the  questionnaires  concerns  itself  with  the 
RDT,  the  other  with  the  GSD.  The  questionnaires  were  structured  to  obtain  the 
evaluation  of  the  prototype  TDWR  by  rating  a  statement  about  each  feature/function 
on  a  five-point  scale  ranging  from  Good  to  Poor,  plus  a  Don't  Know  category  for 
participants  who  did  not  see  a  specific  feature  working.  Comments  were  encouraged. 
(See  appendix  A.) 


PROCEDURE ■ 


The  prototype  TDWR  evaluation  took  place  at  the  Orlando  ATCT.  On  May  5  to 
May  8,  1992,  the  FAA  Technical  Center  test  team  made  an  initial  visit  to  the 
Orlando  ATCT  and  the  MIT/LL  radar  site  to  obtain  specific  information  that  was 
necessary  to  conduct  the  evaluation.  This  information  included  the  number  of 
controller/supervisor  participants,  layout  of  the  tower  cab,  display  equipment, 
duty  schedule  of  participants,  etc.  The  visit  was  coordinated  with  MIT/LL,  the 
PPS ,  and  other  ATCT  personnel. 

The  Test  Director,  FAA  Technical  Center,  ACW-200D,  provided  the  Orlando  PPS  with 
adequate  copies  of  the  supervisor/controller  questionnaires  for  the  evaluation  of 
the  prototype  TDWR.  The  PPS  distributed  and  collected  these  questionnaires  during 
daily  briefings  in  July,  and  then  returned  them  to  the  FAA  Technical  Center. 

ANALYSIS. 

Numerical  values  were  assigned  to  the  questionnaire  responses  with  the  following 
scale:  -2  -  Poor,  -1  -  Fairly  Poor,  0  -  Fair,  +1  -  Fairly  Good,  +2  =  Good,  and 

?  “  Don't  Know.  The  total  number  of  respondents,  the  means  and  the  standard 
deviation  for  the  scaled  items  on  the  GSD  and  the  RDT  questionnaires  are  presented 
in  the  Results  and  Discussion  section.  The  written  responses  from  the  summary  part 
of  the  questionnaire  were  analyzed  using  content  analysis.  These  responses  were 
categorized  and  quantified  to  provide  additional  controller  feedback.  (See 
appendix  B . ) 


RESULTS  AND  DISCUSSION 


The  questionnaires  (appendix  A)  were  designed  to  determine  the  usefulness  of  the 
products  generated  by  the  prototype  TDWR  to  the  ATC  personnel. 

There  were  a  total  of  50  responses  from  the  ATCT  and  the  TRACON  (39  controllers  and 
11  supervisors).  The  high  rate  of  Don't  Know  response  to  the  question  on  both 
the  RDT  and  GSD  questionnaires  concerning  the  rate  of  false  alarms  of  MB  and  GF 
(questions  le  and  If  on  the  RDT,  and  questions  lb  and  Ic  on  the  GSD)  indicates  that 
the  respondents  did  not  understand  the  question  or  did  not  have  enough  information 
to  answer  the  question.  The  low  rating  of  question  lb  and  the  high  rating  of 
question  Ic  on  the  RDT  indicates  that  the  majority  of  the  respondents  prefer  the 
smaller  RDT  for  operational  use.  This  was  the  same  consensus  as  in  the  1991 
evaluation.  The  questions  le  and  If  of  the  GSD  questionnaire  on  the  usefulness  of 
the  storm  motion  and  GF  information  were  rated  as  Good  by  the  majority  of  the 
participants . 

GEOGRAPHICAL  SITUATION  DISPLAY  (GSD). 

The  mean  rating,  standard  deviation,  and  the  number  of  respondents  for  question  1 
of  the  GSD  questionnaire  are  presented  in  table  1.  Overall,  the  GSD  was  rated  in 
the  Fairly  Good  category  (mean  value  of  1.4).  Questions  lb  and  Ic,  rate  of  false 
alarm  of  the  MB  and  GF,  both  had  a  Very  Poor  rating  (mean  value  of  .5  and  .96, 
respectively).  Most  respondents  answered  Don't  Know.  To  determine  the  rate  of 
false  alarms,  the  controller  would  have  to  be  present  at  a  given  display  over  a 
specific  period  of  time.  This  was  not  part  of  the  procedure  for  this 
demonstration. 
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The  following  were  identified  from  the  results  of  the  GSD  questionnaire  (tables  1 
and  2) ; 

1.  Thirty-nine  of  47  respondents  (83  percent)  rated  the  usefulness  of  the  wind 
shift  prediction  (question  la)  as  Fairly  Good  or  above,  and  43  of  46  respondents 
(93  percent)  rated  the  usefulness  of  the  storm  motion  information  (question  If)  as 
Fairly  Good  or  Good. 

2.  Forty  of  47  respondents  (85  percent)  rated  the  usefulness  of  the  displayed  MB 
(question  Id)  as  Fairly  Good  or  better.  Forty-two  of  47  respondents  (89  percent) 
rated  the  displayed  GF  (question  le)  as  Fairly  Good  or  better. 

3.  Most  comments  from  the  respondents  indicate  that  they  are  generally  pleased 
with  the  GSD.  Seven  of  the  9  responses  from  supervisors  (78  percent)  found  the  GSD 
to  be  a  very  useful  tool  in  making  runway  configuration  changes  prior  to  weather 
events  (appendix  B,  GSD  question  3). 


TABLE  1.  QUESTION  1  ON  THE  GSD  QUESTIONNAIRE 


GSD 


Question  1 

Number  of 
Resoondents 

Mean 

Rating 

Standard 

Deviation 

a . 

Usefulness  of  the 
wind  sh'  "  prediction 

47 

1.725 

0.554 

b. 

Rate  of  false  alarm-MB 

47 

0.500 

0.913 

c . 

Rate  of  false  alarm-GF 

46 

0.962 

0.958 

d. 

MB  display  usefulness 

47 

1.568 

0.818 

e . 

GF  display  usefulness 

44 

1.682 

0.740 

f . 

Usefulness  of  the  storm 
motion  information 

46 

1.755 

0.712 

Number  of  respondents,  mean  rating,  and  standard  deviation  of 
the  GSD  questionnaire  (question  1) . 
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TABLE  2. 


GSD  RESPONSES 


Total  GSD  Responses  And  Ratings  For  Question  1 

-2  -  Poor;  -1  -  Fairly  Poor;  0  -  Fa..r;  +1  -  Fairly  Good; 
+2  -  Good;  ?  -  Don't  Know 


1.  Evaluation  Of  The 

Geographical  Situation 
Display  (GSD) 


a.  Usefulness  of  the 
wind  shift  prediction 

b.  Rate  of  false  alarms 
(MB) 

c.  Rate  of  false  alarms 
(GF) 

d.  Usefulness  of  the 
displayed  MB 
information 

e.  Usefulness  of  the 
displayed  GF 
information 

f.  Usefulness  of  the 
displayed  storm  motion 
information 


Rating  Scale 

-2 

-1 

0 

+1 

+2 

7 

1 

0 

2 

7 

32 

5 

1 

1 

8 

10 

3 

24 

1 

0 

6 

11 

8 

20 

1 

0 

3 

9 

31 

3 

1 

0 

1 

8 

34 

3 

1 

0 

1 

5 

38 

1 

Total 

Responses 

47 


47 


46 


47 


47 


46 


RIBBON  DISPLAY  TERMINAL  (RDT) . 

The  mean  rating,  standard  deviation,  and  the  number  of  respondents  to  the  RDT 
questionnaire  appear  in  table  3.  Overall,  the  RDT  was  rated  as  in  the  Fairly  Good 
category  (mean  value  of  1.17). 

The  mean  value  for  question  lb  on  the  RDT  questionnaire  (adequacy  of  the  display 
size  "large")  was  0.422  or  Fair.  This  was  similar  to  the  1991  evaluation  of  the 
same  item.  This  evaluation  reflects  the  general  opinion  of  the  respondents,  that 
of  the  two  RDTs  that  were  tested  (small  size  and  large  size) ,  the  small  RDT  was 
most  preferred. 
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The  following  items  were  identified  as  a  result  of  the  RDT  questionnaire  (tables  3 
and  4) : 

1.  Thirty-nine  of  47  respondents  (83  percent)  rated  the  displayed  MB  information 
(question  Ih)  as  Fairly  Good  or  above,  4  rated  it  as  Fair,  and  4  said  Don't  Know. 
Thirty-six  of  47  respondents  (77  percent)  rated  the  displayed  GF  information 
(question  li)  as  Fairly  Good  or  Good,  2  as  Fair,  and  9  as  Don't  Know. 

2.  Thirty- five  of  47  of  the  respondents  (74  percent)  rated  the  adequacy  of  the 
small  RDT  (12"  x  11"  x  6")  as  Fairly  Good  or  above  for  operational  purposes.  Six 
rated  the  adequacy  as  Fair,  one  as  Fairly  Poor,  two  as  Poor,  and  three  as  Don't 
Know.  On  the  other  hand,  only  26  respondents  (55  percent)  rated  the  large  RDT  as 
Fairly  Good  or  above  for  their  operations.  Seven  rated  the  large  RDT  as  Fair,  2  as 
Fairly  Poor,  10  as  Poor,  and  1  Don't  Know. 

3.  Thirty-nine  of  46  respondents  (85  percent)  rated  the  daytime  readability  of  the 
"small”  RDT  (question  Ic)  as  Fairly  Good  or  above,  3  as  Fair,  and  4  rated  it  as 
Don't  Know.  Thirty-seven  of  46  respondents  (80  percent)  rated  the  nighttime 
readability  of  the  "small”  RDT  (question  Id)  as  Fairly  Good  or  above,  3  as  Fair, 
and  6  as  Don't  Know. 

4.  Thirty-eight  of  47  respondents  (81  percent)  rated  the  timeliness  of  the 
displayed  data  (question  Ig)  as  Fairly  Good  or  better,  3  as  Fair,  and  6  as  Don't 
Know. 

5.  Five  of  the  18  comments  from  the  respondents  indicate  that  they  would  like  to 
see  the  centerfield  wind  displayed  in  the  upper  lef*"  corner  of  the  display  rather 
than  in  the  lower  part  of  the  display  (appendix  B,  RDT  question  3) .  No  reason  was 
given  for  this  preference. 
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TABLE  3.  QUESTION  1  ON  THE  RDT  QUESTIONNAIRE 


RDT 


Question  1 

Number  of 
Resnondents 

Mean 

Ratine 

Standard 

Deviation 

a . 

Adequacy  of  small  RDT 

47 

1.250 

1.080 

b. 

Adequacy  of  large  RDT 

46 

0.422 

1.545 

c . 

Daytime  readability 
of  small  RDT 

46 

1.548 

0.633 

d. 

Nighttime  readability 
of  small  RDT 

46 

1.575 

0.636 

e . 

MB  rate  of  false  alarm 

47 

0.529 

1.790 

f . 

GF  rate  of  false  alarm 

46 

0.563 

1.153 

g- 

Timeliness  of  data 

47 

1.415 

0.631 

h. 

Usefulness  of  the  MB 
displayed  information 

47 

1.604 

0.660 

i . 

Usefulness  of  the  GF 
displayed  information 

47 

1.658 

0.582 

Number  of  respondents,  mean  rating,  and  standard  deviation  of 
the  RDT  questionnaire  (question  1) 
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TABLE  4. 


RDT  RESPONSES 


CONCLUSIONS 


The  evaluation  of  the  Massachusetts  Institute  of  Technology/Lincoln  Laboratory 
(MIT/LL)  prototype  Terminal  Doppler  Weather  Radar  (TDWR)  by  air  traffic  control 
(ATC)  personnel  at  the  Orlando  International  Airport  (MCO) ,  Orlando,  Florida,  has 
provided  input  as  to  the  weather  warnings,  readability  and  adequacy  of  the  Ribbon 
Display  Terminal  (RDTs) ,  adequacy  of  the  RDT  size,  and  the  storm  movement  and  gust 
front  (GF)  information  on  the  Geographical  Situation  Display  (GSD) .  The  evaluation 
was  unable  to  give  any  useful  information  regarding  the  rate  of  false  alarms.  This 
may  have  to  be  evaluated  in  the  MIT/LL  final  report  on  the  1992  system.  Some  of 
the  significant  findings  are; 

1.  The  ATC  generally  liked  the  system  and  found  it  helpful  in  their  operations. 

2.  Supervisors  found  the  GSD  helpful  in  making  runway  configurations  prior  to 
weather  events . 

3.  Ninety- three  percent  of  the  respondents  rated  the  usefulness  of  the  displayed 
storm  motion  information  (GSD  question  If)  as  Fairly  Good  or  above,  and  89  percent 
rated  the  usefulness  of  the  displayed  GF  information  (GSD  question  le)  as  Fairly 
Good  or  Good.  These  were  two  of  the  highest  ratings  on  the  questionnaire. 

4.  The  rate  of  false  alarms  could  not  be  evaluated  by  the  ATC  personnel  with  the 
limited  information  available  to  them  during  the  demonstration. 

5.  The  size  of  the  RDT  and  the  location  of  the  centerfield  is  important  to  the 
controllers.  Six  of  18  comments  were  about  the  size  of  the  RDT.  The  respondents 
generally  preferred  the  small  RDT  (12"  x  11  1/4"  x  6")  over  the  large  RDT  (15"  x 
15"  X  6")  and  most  felt  that  the  centerfield  should  be  at  the  top  of  the  display. 


RECOMMENDATIONS 


The  Orlando  air  traffic  control  (ACT)  personnel  have  accepted  the  prototype 
Terminal  Doppler  Weather  Radar  (TDWR)  as  a  functional  radar  system  capable  of 
meeting  their  operational  needs.  However,  two  minor  problems  that  should  be 
considered  are:  (1)  the  preference  of  the  small  Ribbon  Display  Terminal  (RDT)  to 
the  large,  and  (2)  the  location  of  the  centerfield  winds  on  the  RDT.  These 
concerns  may  be  resolved  with  the  Raytheon  production  TDWR. 


It  is  recommended  that  a  user  evaluation  of  the  production  TDWR  prior  to  the 
decision  for  full  deployment  be  conducted,  and  if  any  major  problems  occur,  that 
they  be  addressed  at  that  time. 


ATC 

ATCT 

CIP 

FAA 

GF 

GSD 

LL 

LLWAS -  3 

MB 

MBA 

MCO 

MIT 

PPS 

RDT 

TDWR 

TRACON 


ACRONYMS  AND  ABBREVIATIONS 

Air  Traffic  Control 
Air  Traffic  Control  Tower 
Capital  Investment  Plan 
Federal  Aviation  Administration 
Gust  Front 

Geographic  Situation  Display 

Lincoln  Laboratory 

Low  Level  Wind  Shear  Alert  System 

Microburst 

Microburst  Alert 

Orlando  International  Airport 

Massachusetts  Institute  of  Technology 

Plans  and  Procedure  Specialist 

Ribbon  Display  Terminal 

Terminal  Doppler  Weather  Radar 

Terminal  Radar  Approach  Control 
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APPENDIX  A 

TDWR  EVALUATION  QUESTIONNAIRES 


RDT 


TOWER _ TRACON _ 

CONTROLLER _  SUPERVISOR _ 

Please  rate  the  TDWR  using  the  following  scale: 

-2  -  Poor;  -1  -  Fairly  Poor;  0  -  Fair;  +1  -  Fairly  Good; 
+2  -  Good;  ?  -  Don't  Know 


1 .  Evaluation  Of  The  Ribbon 
Display  Terminal  (RDT) 


a.  Adequacy  of  display  size 
"small" 

b.  Adequacy  of  display  size 
"large" 

c.  Daytime  readability  of 
"small"  RDT 

d.  Nighttime  readability  of 
"small"  RDT 

e.  Rate  of  false  alarms  (MB) 


f.  Rate  of  false  alarms  (GF) 


g.  Timeliness  of  displayed 
data 

h.  Usefulness  of  the  displayed 
MB  information 

i.  Usefulness  of  the  displayed 
GF  informat’ ~n 

2.  Please  state  instances  (if  any)  of  wind  shear  that  the  system  did 
not  display: 

a.  Microburst  - , 

b.  Gust  Front  - — . 

3.  Please  provide  comments  on  any  rating  of  0  or  lower  and/or  any  other 
comments  on  the  role  of  the  RDTs  (Ribbon  Display  Terminals) . 


I - 1 


Rating  Scale  | 

_ 1 

1 

-2  1-1 

1 

1 

0 

1  1  1 

+1  1+2  1  ?  1 

1  1  1 

1 

1 

1  j  1 

1  1  1 

1  1  1 

1 

1 

1 

1  1  1 

1  1  1 

1  1  1 

1  1  1 

1 

1 

1 

1 

1  1  1 

1  1  1 

1  1  1 

1  1  1 

1 

1 

1 

1  1  \ 

1  1  1 

1  1  1 

1  1  1 

1 

1 

.  ..  1  . . 

1  1  1 

1  1  1 

1  1  1 

1  1  1 

1 

1 

1  1  1 

1  1  1 

1  1  1 

1 

1 

t  1  1 

1  1  1 

1  1  1 

1 

1 

1 

1  )  I 

1  1  1 

1  1  1 

1  1  1 

1 

1 

1 

)  1  1 

^ _ 1 _ 1 

USE  BACK  OF  PAGE  IF  NECESSARY 
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GSD 


TOWER _  TRACON _ 

CONTROLLER _  SUPERVISOR _ 

Please  rate  the  TDWR  using  the  following  scale; 

-2  -  Poor;  -1  -  Fairly  Poor;  0  -  Fair;  +1  -  Fairly  Good; 
+2  -  Good;  ?  -  Don't  Know 


1 .  Evaluation  Of  The 

Geographical  Situation 
Display  (GSD) 

a.  Usefulness  of  the  wind  shift 
prediction 

b.  Rate  of  false  alarms  (MB) 


c.  Rate  of  false  alarms  (GF) 


d.  Usefulness  of  the  displayed 
MB  information 

e.  Usefulness  of  the  displayed 
GF  information 

f.  Usefulness  of  the  displayed 
storm  motion  information 

2.  Please  state  instances  (if  any)  of  wind  shear  that  the  system  did 
not  display: 

a.  Microburst  - , 

b.  Gust  Front  - . 

3.  Supervisors  only;  Was  the  GSD  useful  in  making  runway  configurations 
changes  prior  to  weather  events?  Yes  or  No.  Please  explain. 


t  I 


'  Rating  Scale  j 

_ 1 

I  I - 1  1 - 1  1 

-2  1-1  1  0  1+1  1+2 
till 

I 

7  1 

T  1  I  1 

ill! 

1  I  1  1 

III 
_ \ _ ! _ ! _ \ _ 

_ 1 

k.  Please  provide  comments  on  any  rating  of  0  or  lower  and/or  any  other 
comments  on  the  Geographical  Situation  Displays  (GSD) . 


USE  BACK  OF  PAGE  IF  NECESSARY 
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APPENDIX  B 


QUESTIONNAIRE  COMMENTS 


Comments  from  the  GSD  questionnaire: 

GSD  Question  2.  Please  state  instances  (if  any)  of  wind  shear  that  the  system  did 
not  display:  a.  Microburst;  b.  Gust  Front. 

Of  the  nine  responses  to  this  question,  the  answers  were  as  follows: 

a.  One  answer  of  "not  applicable"  to  both  "a"  and  "b"  parts. 

b.  Six  respondents  answered  with  a  question  mark  (?)  to  both  "a"  and  "b"  parts. 

c.  One  answer  was  "Pilots  reported  wind  shear  when  not  detected." 

d.  One  answer  was  "More  often  than  not." 

It  would  appear  from  these  responses  that  the  respondents  either  did  not  understand 
the  question  or  did  not  have  enough  information  to  give  a  definitive  answer. 

GSD  Question  3.  supervisors  only.  Was  the  GSD  useful  in  making  runway  configuration 
changes  prior  to  weather  events?  Yes  or  No. 

Of  the  nine  responses  to  this  question  to  the  supervisors,  the  responses  were  as 
follows : 

a.  Seven  replied  affirmative  or  yes  to  the  question. 

b.  One  replied  "Not  applicable." 

c.  One  replied  "How  would  you  know  if  it  were  a  false  alarm  or  not?" 

The  majority  of  the  responses  to  this  question  seem  to  indicate  that  the  GSD  was 
useful  in  making  the  runway  configuration.  The  two  questionable  responses  may  have 
come  from  someone  that  did  not  have  access  to  the  GSD  (controllers  rather  than 
supervisors) . 

GSD  Question  4 .  Please  provide  comments  or  any  rating  of  0  or  lower  and/or  any  other 
comments  on  the  Geographical  Situation  Display  (GSD) . 

1.  "The  system  is  very,  very  unreliable.  It  seems  to  go  out  every  time  the  bad 
Wx  moves  in." 

2.  "Total  system  seems  to  always  fail  when  severe  weather  is  around." 

3.  "This  is  very  useful  and  effective  equipment." 

4.  "Do  not  care  for  the  "downgraded"  version  which  lacks  movement  speeds,  etc.  LLWAS 
arrows  not  necessary . -USEFUL  not  as  good." 

5.  "Good  system." 

6.  "Excellent  system." 

7.  "What  constitutes  a  false  alarm." 
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8.  "Wind  speed  red  circles  are  too  small  and  not  readable  when  you  are  on  a 
30-50  mile  range." 

9.  "This  year  in  particular  (as  opposed  to  years  past)  seem  to  be  more  false  MBA 
&  WSA  (possible  due  to  new  algorithms)." 

10.  "It  has  seemed  that  when  it  is  most  needed  the  system  goes  down." 

11.  "Excellent,  well  worth  the  $.  Reliability  could  be  better.  Why,  why  are  you 
going  to  dismantle  this  equipment." 

12.  "It  would  be  nice  to  have  a  window  feature  in  the  corner  of  the  GSD  crt  for  the 
purpose  of  zooming  in  on  a  particular  area  of  Wx.  This  way  you  could  stay  on 
the  big  range  for  the  overall  picture  and  get  detailed  information  for  a 
specific  Wx  area.  Need  some  type  of  graphic  anti-overlap  so  that  the  storm 
track  and  gust  front  graphic  don't  obscure  each  other.  That  was  a  problem  on 
several  occasions . " 

13.  "Very  good  system. .. especially  the  storm  motion  info  (since  the  ASR9's  weather 
is  almost  as  good  for  seeing  the  location  of  the  Wx) .  I  would  like  to  see  other 
range  possibilities,  other  than  just  15  miles,  30  miles,  etc.  Possibly  a 
"window"  somewhat  like  AutoCad,  that  can  be  positioned  with  the  trackball  for 
a  closeup  of  the  actual  storm  and  its  relation  to  the  airport." 

Comments  from  the  RDT  questionnaire: 

RDT  Question  2.  Please  state  the  instances  (if  any)  of  wind  shear  that  the  system 
did  not  display;  a.  Microburst;  b.  Gust  Front. 

The  responses  to  this  question  were  as  follows; 

a.  Seven  respondents  gave  question  marks  i"?")  as  their  answer  to  the  question 
(both  parts  "a"  and  "b"). 

b.  One  respondent  answered  "None." 

The  responses  from  the  supervisors/controllers  seem  to  indicate  that  they  could  not 
determine  if  and  when  the  RDT  did  not  display  a  Microburst  or  Gust  Front.  The  reason 
may  be  the  lack  of  information  to  make  this  determination  or  the  lack  of  understanding 
the  system. 

RDT  Question  3.  Please  provide  comments  on  any  rating  of  "0"  or  lower  and/or  any 
other  comments  on  the  role  of  the  RDT's  (Ribbon  Display  Terminals). 

1.  "Displays  are  too  large." 

2.  "A,B  -  Both  displays  are  too  large  and  in  an  unsafe  location,  blocking  the  view 
of  the  runways  when  in  a  seated  position." 

3.  "Several  times  the  RDT  was  showing  a  "MBA"  but  did  not  have  a  aural  alarm." 

4.  "The  large  RDT  is  too  large." 

5 .  "Did  not  use . " 
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6.  "RDT's  should  be  made  to  fit  into  existing  LLWAS  cavity.  Centerfield  wind  should 
be  in  the  upper  left  corner." 

7.  "Centerfield  wind  should  be  on  the  top  line  rather  than  the  bottom  line." 

8.  "Excellent  system." 

9.  "Not  sure  of  the  rate  of  false  alarm." 

10.  "Approach  end  and  departure  end  information  displays  together." 

11.  "Keep  rwys  together;  i.e.  arrival,  dept  end  of  the  same  rwy  together  --  don't 
split  them  up." 

12.  "Move  the  centerfield  wind  back  to  the  top  of  the  Ribbon  Display." 

13.  "Severe  weather  causes  a  "shut  down"  --  too  much  delay." 

14.  "Reliability  could  be  better  -  excellent  piece  of  equipment,  but,  why  does  it 
cost  so  much?  I  feel  that  the  RDTs  cost  wav  too  much." 

15.  "RDTs  should  utilize  economical  off  the  shelf  hardware  to  reduce  cost  and 
facilitate  repairs.  The  exotic  RDT  used  during  testing  is  cost  prohibitive  and 
once  in  production  there  may  be  problems  getting  spare  parts.  RDT  should  be  dual 
purpose  so  they  could  replace  LLWAS  display  and  display  LLWAS  info  when  doppler 
was  not  operational . " 

16.  "Centerfield  wind  should  be  in  the  upper  left.  Why  can't  they  use  regular  small 
TV  monitors?  I  understand  your  Ribbon  Displays  cost  $7000.00  -  that's 
ridiculous . " 

17.  "Display  too  large.  Centerfield  wind  indicator  should  be  moved  to  the  upper  part 
of  display." 

18.  "Large  display  is  way  too  cluttersome . " 

The  comments  on  question  4  are  self  explanatory,  however  the  following  should  be 

noted: 

a. 


b. 


c . 


The  largest  number  of  comments  (6  of  18)  were  comments  stating  that  the  RDTs 
were  too  large . 

Five  of  the  comments  were  comments  on  the  centerfield  winds.  Most 
respondents  wanted  the  centerfield  wind  indicator  moved  to  the 
top  of  the  RDT's  display. 

Three  of  the  respondents  questioned  the  reliability  of  the  system  and  three 
thought  the  system  was  too  expensive. 
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